Papers with hallucination detection methods
Benchmarking LLM Faithfulness in RAG with Evolving Leaderboards (2025.emnlp-industry)
Copied to clipboard
Manveer Singh Tamber, Forrest Sheng Bao, Chenyu Xu, Ge Luo, Suleman Kazi, Minseok Bae, Miaoran Li, Ofer Mendelevitch, Renyi Qu, Jimmy Lin
| Challenge: | Large language models (LLMs) excel in various tasks, but often produce hallucinations . retrieved contexts, misrepresent information, or generate outright contradictions . |
| Approach: | They propose a framework that measures hallucination faithfulness of large language models . they introduce a leaderboard that leverages diverse human-annotated hallucinian examples . |
| Outcome: | The proposed framework improves hallucination evaluations by leveraging human-annotated examples. |
DelucionQA: Detecting Hallucinations in Domain-specific Question Answering (2023.findings-emnlp)
Copied to clipboard
Mobashir Sadat, Zhengyu Zhou, Lukas Lange, Jun Araki, Arsalan Gundroo, Bingqing Wang, Rakesh Menon, Md Parvez, Zhe Feng
| Challenge: | Hallucination is a well-known phenomenon in text generated by large language models . state-of-the-art LLMs still have a number of weaknesses, including the tendency to generate hallucinatory statements without considering the factuality . |
| Approach: | They propose a dataset that captures hallucinations made by retrieval-augmented LLMs . they propose to use these methods to help detect hallucinosity in QA tasks . |
| Outcome: | The proposed method captures hallucinations made by retrieval-augmented LLMs for QA tasks. |
FactSelfCheck: Fact-Level Black-Box Hallucination Detection for LLMs (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods to detect hallucinated content are limited by their tendency to generate factual errors. |
| Approach: | They propose a black-box sampling-based method that enables fine-grained fact-level detection by representing text as interpretable knowledge graphs consisting of facts in the form of triples. |
| Outcome: | The proposed method improves hallucination correction by 35.5% compared to baseline methods while sentence-level SelfCheckGPT yields only 10.6% improvement. |
When Models Lie, We Learn: Multilingual Span-Level Hallucination Detection with PsiloQA (2025.findings-emnlp)
Copied to clipboard
Elisei Rykov, Kseniia Petrushina, Maksim Savkin, Valerii Olisov, Artem Vazhentsev, Kseniia Titova, Alexander Panchenko, Vasily Konovalov, Julia Belikova
| Challenge: | Existing hallucination detection benchmarks operate at the sequence level and are limited to English . Existing methods lacking fine-grained, multilingual supervision are limited in English based on the sequence . |
| Approach: | They propose a large-scale, multilingual dataset annotated with span-level hallucinations across 14 languages. |
| Outcome: | The proposed dataset annotated with span-level hallucinations across 14 languages is scalable and cost-efficient. |
ICR Probe: Tracking Hidden State Dynamics for Reliable Hallucination Detection in LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for hallucination detection rely on static and isolated representations, overlooking their dynamic evolution across layers. |
| Approach: | They propose a method which captures the cross-layer evolution of hidden states and propose 'ICR Probe' which capture the evolution of the hidden states. |
| Outcome: | The proposed method achieves superior performance with significantly fewer parameters and ablation studies offer deeper insights into the underlying mechanism of the method, improving its interpretability. |
ReFL: Reflective Feedback Learning for Hallucination Detection of Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for detecting hallucinations depend on external knowledge sources, incurring high computational costs and limiting real-time applicability, or extract the model’s internal states, leading to poor generalization. |
| Approach: | They propose a hallucination detection framework that leverages corrective in-context learning to guide LLMs to recognize their own prediction errors and adjust internal representations, critically without updating model weights. |
| Outcome: | The proposed framework outperforms existing methods on two benchmark datasets and achieves state-of-the-art performance. |
Hallucination Detection for Generative Large Language Models by Bayesian Sequential Estimation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for detecting hallucinations require large numbers of observations to be retrieved, increasing response times. |
| Approach: | They propose a framework that leverages Bayesian sequential analysis to optimize the trade-off between costs and benefits during the hallucination detection process. |
| Outcome: | The proposed framework surpasses existing methods in efficiency and precision of hallucination detection. |
Prompt-Guided Internal States for Hallucination Detection of Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) generate incorrect or logically incorrect responses, which is known as LLM hallucinations. |
| Approach: | They propose a framework for supervised hallucination detection using in-domain data by prompting changes to the structure related to text truthfulness in LLMs’ internal states. |
| Outcome: | The proposed framework enhances the cross-domain generalization of existing hallucination detection methods. |
VIB-Probe: Detecting and Mitigating Hallucinations in Vision-Language Models via Variational Information Bottleneck (2026.acl-long)
Copied to clipboard
| Challenge: | Existing hallucination detection methods rely on external verification tools . however, entanglement of visual-linguistic syntax and noise makes it difficult to detect hallucis . |
| Approach: | They propose a hallucination detection framework that leverages the Variational Information Bottleneck theory to detect hallucinic heads and to infer hallucication mitigation strategies. |
| Outcome: | The proposed framework outperforms baselines in hallucinations and noise detection environments. |
The Illusion of Progress: Re-evaluating Hallucination Detection in LLMs (2025.emnlp-main)
Copied to clipboard
Denis Janiak, Jakub Binkowski, Albert Sawczyn, Bogdan Gabrys, Ravid Shwartz-Ziv, Tomasz Jan Kajdanowicz
| Challenge: | Large language models (LLMs) have revolutionized natural language processing, but their tendency to hallucinate poses serious challenges for reliable deployment. |
| Approach: | They propose to use ROUGE to assess lexical overlap to determine accuracy of hallucination detection methods. |
| Outcome: | The proposed evaluation frameworks can rival complex methods, exposing a fundamental flaw in current evaluation practices. |
MARCH: Multi-Agent Reinforced Check for Hallucination (2026.acl-long)
Copied to clipboard
Zhuo Li, Yupeng Zhang, Pengyu Cheng, Jiajun Song, Mengyu Zhou, Hao Li, Shujie Hu, Yu Qin, null Erchao.zec, Xiaoxi Jiang, null Guanjunjiang
| Challenge: | Existing methods to detect hallucinations suffer from inherent confirmation bias, where the verifier inadvertently reproduces the errors of the original generation. |
| Approach: | They propose a framework that enforces rigorous factual alignment by leveraging deliberate *information asymmetry* by combining a pipeline of three specialized agents: a Solver, a Proposer, and a Checker. |
| Outcome: | Extensive experiments across hallucination benchmarks demonstrate that MARCH substantially reduces hallucinism rates. |
Detecting Hallucinations in SpeechLLMs at Inference Time Using Attention Maps (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for hallucination detection for text-based LLMs do not capture audio-specific signals. |
| Approach: | They propose to capture pathological attention patterns associated with hallucination using four attention-derived metrics to train lightweight logistic regression classifiers. |
| Outcome: | The proposed approach outperforms baselines on in-domain data and generalises to out-of-domain ASR settings. |